NASA/CR-201 3-21 7911 



Range-specific High-resolution Mesoscale 
Model Setup 


Leela R. Watson 

ENSCO, Inc., Cocoa Beach, Florida 

NASA Applied Meteorology Unit, Kennedy Space Center, Florida 


January 2013 


NASA STI Program ... in Profile 


Since its founding, NASA has been dedicated 
to the advancement of aeronautics and space 
science. The NASA scientific and technical 
information (STI) program plays a key part in 
helping NASA maintain this important role. 

The NASA STI program operates under the 
auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 
disseminates NASA’s STI. The NASA STI 
program provides access to the NASA 
Aeronautics and Space Database and its public 
interface, the NASA Technical Reports Server, 
thus providing one of the largest collections of 
aeronautical and space science STI in the world. 
Results are published in both non-NASA channels 
and by NASA in the NASA STI Report Series, 
which includes the following report types: 

• TECHNICAL PUBLICATION. Reports of 
completed research or a major significant 
phase of research that present the results of 
NASA Programs and include extensive data 
or theoretical analysis. Includes compila- 
tions of significant scientific and technical 
data and information deemed to be of 
continuing reference value. NASA counter- 
part of peer-reviewed formal professional 
papers but has less stringent limitations on 
manuscript length and extent of graphic 
presentations. 

• TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 
preliminary or of specialized interest, 
e.g., quick release reports, working 
papers, and bibliographies that contain 
minimal annotation. Does not contain 
extensive analysis. 

• CONTRACTOR REPORT. Scientific and 
technical findings by NASA-sponsored 
contractors and grantees. 


• CONFERENCE PUBLICATION. 

Collected papers from scientific and 
technical conferences, symposia, seminars, 
or other meetings sponsored or 
co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 
technical, or historical information from 
NASA programs, projects, and missions, 
often concerned with subjects having 
substantial public interest. 

• TECHNICAL TRANSLATION. 
English-language translations of foreign 
scientific and technical material pertinent to 
NASA’s mission. 

Specialized services also include organizing 
and publishing research results, distributing 
specialized research announcements and 
feeds, providing information desk and personal 
search support, and enabling data exchange 
services. 

For more information about the NASA STI 
program, see the following: 

• Access the NASA STI program home page 
at http://www.sti.nasa.gov 

• E-mail your question to help@sti.nasa.gov 

• Fax your question to the NASA STI 
Information Desk at 443-757-5803 

• Phone the NASA STI Information Desk at 
443-757-5802 

• Write to: 

STI Information Desk 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 


NASA/CR-2013-21791 1 



Range-specific High-resolution Mesoscale 
Model Setup 


Leela R. Watson 

ENSCO, Inc., Cocoa Beach, Florida 

NASA Applied Meteorology Unit, Kennedy Space Center, Florida 


National Aeronautics and 
Space Administration 

Kennedy Space Center 

Kennedy Space Center, FL 32899-0001 


January 2013 


Acknowledgements 


The author thanks Mr. William Roeder of the 45th Weather Squadron and Mr. Theodore Wilz of 
the Wallops Flight Facility for providing input and valuable feedback on the overall direction of 
this task, and Dr. William Bauman, Ms. Winifred Crawford and Ms. Jaclyn Shafer of the Applied 
Meteorology Unit for their assistance in the completion of this task. 


Available from: 


NASA Center for AeroSpace Information 
7115 Standard Drive 
Hanover, MD 21076-1320 
443-757-5802 


This report is also available in electronic form at 

http://science.ksc.nasa.gov/amu/ 


Executive Summary 

Mesoscale weather conditions can have an adverse effect on space launch, landing, ground 
processing, and weather advisories, watches, and warnings at the Eastern Range (ER) in 
Florida and Wallops Flight Facility (WFF) in Virginia. During summer, land-sea interactions 
across Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS) lead to 
sea breeze front formation, which can spawn deep convection that can hinder operations and 
endanger personnel and resources. Many other weak locally-driven low-level boundaries and 
their interactions with the sea breeze front and each other can also initiate deep convection in 
the CCAFS/KSC area. These convective processes often last 60 minutes or less and pose a 
significant challenge to the local forecasters. Surface winds during the transition seasons 
(spring and fall) pose the most difficulties for the forecasters at WFF. They also encounter 
problems forecasting convective activity and temperature during those seasons. Therefore, 
accurate mesoscale model forecasts are needed to better forecast a variety of unique weather 
phenomena. Global and national scale models cannot properly resolve important local-scale 
weather features at each location due to their horizontal resolutions being much too coarse. 
Therefore, a properly tuned model at a high resolution is needed to provide improved capability. 
To accomplish this, the ER and WFF supported the tasking of the Applied Meteorology Unit 
(AMU) to perform a number of sensitivity tests in order to determine the best model 
configuration for operational use at each of the ranges to best predict winds, precipitation, and 
temperature. 

The AMU compared model forecasts for both the ER and WFF using different Weather 
Research and Forecasting (WRF) model dynamical cores, grid configurations, and physical 
schemes to determine the impact on model skill. Different model configurations were tested by 
varying the dynamical core, grid spacing, domain size, and forecast length. This enabled the 
AMU to determine the optimal configuration that allowed for the largest domain size and highest 
resolution to capture unique weather phenomena with the shortest wall-clock run time. Once the 
configurations were chosen, the AMU varied the model physics to determine which produced 
the best forecasts. The WRF model forecasts were validated using simple statistics that 
compared locally available observational surface and upper air data to the forecast data. The 
objective statistical analysis included the model bias, mean error, and the root mean square 
error. Precipitation forecasts were compared to nationally available rainfall data using the 
Method for Object-Based Diagnostic Evaluation (MODE), a technique developed at the National 
Center for Atmospheric Research. 

The AMU ran test cases in the warm and cool seasons at the ER and for the spring and fall 
seasons at WFF. For both the ER and WFF, the Advanced Research WRF core outperformed 
the Non-hydrostatic Mesoscale Model core. Results for the ER indicate that the Lin 
microphysical scheme and the Yonsei University (YSU) planetary boundary layer (PBL) scheme 
is the optimal model configuration for the ER. It consistently produced the best surface and 
upper air forecasts, while performing fairly well for the precipitation forecasts. Both the Ferrier 
and Lin microphysical schemes in combination with the YSU PBL scheme performed well for 
WFF in the spring and fall seasons. 

Future work under the multi-year AMU modeling tasks will provide a recommended local 
data assimilation and numerical forecast model design optimized for both the ER and WFF to 
support space launch activities. The AMU will determine the best software and type of 
assimilation to use, as well as determine the best grid resolution for the initialization based on 
spatial and temporal availability of data and the wall clock run-time of the initialization. The AMU 
will transition from the WRF EMS to NU-WRF, a NASA-specific version of the WRF that takes 
advantage of unique NASA software and datasets. 
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1. Introduction 

Mesoscale weather conditions can have an adverse effect on space launch, landing, ground 
processing, and weather advisories, watches, and warnings at the Eastern Range (ER) in 
Florida and Wallops Flight Facility (WFF) in Virginia. During summer, land-sea interactions 
across Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS) lead to 
sea breeze front formation, which can spawn deep convection that can hinder operations and 
endanger personnel and resources. Many other weak locally-driven low-level boundaries and 
their interactions with the sea breeze front and each other can also initiate deep convection in 
the CCAFS/KSC area. Some of these other boundaries include the Indian River Breeze front, 
Banana River Breeze front, outflows from previous convection, horizontal convective rolls, 
convergence lines from other inland bodies of water such as Lake Okeechobee, the Trailing 
Convergence Line from convergence of sea breeze fronts due to the shape of Cape Canaveral, 
frictional convergence lines from the Bahamas Islands, convergence lines from soil moisture 
differences, convergence lines from cloud shading, and others. All these subtle weak boundary 
interactions often make forecasting of operationally important weather very difficult at 
CCAFS/KSC during the summer. These convective processes often build quickly, last a short 
time (60 minutes or less), and occur over small distances, all of which also poses a significant 
challenge to the local forecasters. Surface winds during the transition seasons (spring and fall) 
pose the most difficulties for the forecasters at WFF. They also encounter problems forecasting 
convective activity and temperature during those seasons. Therefore, accurate mesoscale 
model forecasts are needed to aid in their decision making. 

Both the ER and WFF would benefit greatly from high-resolution mesoscale model output to 
better forecast a variety of unique weather phenomena. Global and national scale models 
cannot properly resolve important local-scale weather features at each location due to their 
horizontal resolutions being much too coarse. Therefore, a properly tuned model at a high 
resolution is needed to provide improved capability. This task is a multi-year study in which the 
Weather Research and Forecasting (WRF) model will be tuned individually for each range. The 
goal of the first year, the results of which are in this report, was to tune the WRF model based 
on the best model resolution and run time while using reasonable computing capabilities. To 
accomplish this, the ER and WFF supported the tasking of the Applied Meteorology Unit (AMU) 
to perform a number of sensitivity tests in order to determine the best model configuration for 
operational use at each of the ranges to best predict winds, precipitation, and temperature. 

1.1 Background Information 

A number of studies have been conducted on model forecast sensitivities to different 
parameters. Most studies were conducted for a specific area or particular meteorological 
phenomena. The importance of a comprehensive and accurate analysis of the state of the 
atmosphere for the forecast initial conditions is vital to improving model skill. Cheng et al. (2010) 
notes that mesoscale weather phenomena are very sensitive to the atmospheric physical 
processes and that model physics schemes should be carefully evaluated when setting up a 
forecasting system. Their study found that, although the model physics is important, it played a 
secondary role to the synoptic environment, which must be well simulated in order to produce a 
good mesoscale forecast. 

It is well known that warm season rainfall is very challenging to predict (Olson et al. 1995). A 
study conducted by Jankov et al. (2005) found that the choice of a microphysical scheme 
impacted the total rain volume of warm season convection. Gallus and Bresch (2006) extended 
these results and found that: 
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• forecasted peak summer rain rates over the United States were more sensitive to 
the physics package than to the dynamical core, 

• total rain volume was more sensitive to the dynamical core than the physics, and 

• both the physics and dynamical core had a greater impact on the forecasts than the 
initial conditions that were chosen. 

1.2 Mesoscale Model Core Options 

The WRF numerical weather modeling system is a next-generation mesoscale forecast 
model that can be used for a wide range of applications with an emphasis on horizontal grid 
sizes in the range of 1-10 km (Tao et al. 2011). It consists of two dynamical cores, Advanced 
Research WRF (ARW) and Non-hydrostatic Mesoscale Model (NMM). The ARW core was 
developed primarily at the National Center for Atmospheric Research (NCAR) while the NMM 
was developed at the National Centers for Environmental Prediction (NCEP). The work 
described in this report employed the WRF Environmental Modeling System (EMS) software, 
which was developed by the National Weather Service (NWS) Science and Operations Officer 
(SOO) Science and Training Resource Center (STRC; 
http://strc.comet.ucar.edu/software/newrems) . A benefit of using the WRF EMS is that it 
incorporates both dynamical cores into a single end-to-end forecasting model (Rozumalski 
2006). The software consists of pre-compiled programs that are easy to install and run. 

For this study, the AMU compared forecasts from both the ARW and NMM cores of the 
WRF model. The ARW core is a fully compressible, non-hydrostatic mesoscale model with a 
hydrostatic option. It consists of a mass-based hydrostatic pressure terrain following coordinate, 
Arakawa C-grid staggering for the horizontal grid, time-split integration using a third order 
Runge-Kutta scheme with a small step for acoustic and gravity wave modes, and up to sixth 
order advection options in the horizontal and vertical (Skamarock et al. 2005). There are also 
full physics options for microphysics, planetary boundary layer, cumulus parameterization, 
radiation, and land surface schemes (Skamarock et al. 2005). The NMM core is also a fully 
compressible, non-hydrostatic mesoscale model with a hydrostatic option (Janjic et al. 2001, 
Janjic 2003a, b). It consists of a hybrid sigma-pressure, terrain following vertical coordinate, 
Arakawa E-grid, a forward-backward time integration scheme, a second order advection option 
in the horizontal and vertical, and conservation of energy and enstrophy (Janjic 1984). Most 
physics packages available to the ARW are also available to the NMM, but a majority of them 
have not been tested or have undergone preliminary testing only with the NMM. Therefore, the 
physics options for the NMM are more limited than for the ARW. 

1.3 Model Physics Options 

The WRF model offers a choice of many different physics options including schemes for 
boundary layer, surface layer, radiation, microphysics, turbulence, and cumulus 
parameterization. For this task, different microphysical and planetary boundary layer schemes 
were compared since they offered the most varied options. While a full comparison of all the 
physics would have been ideal, time constraints limit the comparison to a select group that, 
based on a literature review, are the most widely used within the modeling community and/or 
have been shown to produce the best results. The following provides a brief description of each 
of the model physics options that were tested in this task. 

1.3.1 Microphysical Schemes 

Microphysics schemes are important for accurately modeling cloud process to correctly 
simulate the state of the atmosphere. Microphysics schemes in the WRF model explicitly 
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resolve various forms of water at a grid point including vapor, cloud water, cloud ice, rain, snow, 
and hail. Some of the microphysics schemes available in WRF include all of these forms, while 
others exclude some of them. Three microphysical algorithms that include all forms of water 
were tested in this task and are described below: Lin, Ferrier, and WDM6. 

The Lin scheme is based on Lin et al. (1983), Rutledge and Hobbs (1984), and Chen and 
Sun (2002) and is a two-dimensional scheme. It predicts cloud water, cloud ice, water vapor, 
rain, snow, and graupel at each grid point based on advection, production, and fallout. It uses 
the “bulk water” parameterization technique to represent precipitation fields that are all assumed 
to follow exponential size distribution functions (Lin et al. 1983). It is considered a sophisticated 
microphysics scheme in WRF and is suitable for real-data high-resolution simulations. It was 
added to the WRF model in 2000. 

The Ferrier scheme predicts changes in water vapor and total condensate in the forms of 
cloud water, cloud ice, rain, and precipitation ice (snow, graupel, and sleet). The model advects 
the water vapor and total condensate, which is made up of the individual hydrometeor fields, 
making it computationally efficient. The Ferrier scheme also uses the bulk water 
parameterization technique. It is used in the operational NCEP models and was added to the 
WRF model in 2000. 

The WRF Double-Moment 6-Class (WDM6) scheme was developed by Lim and Hong 
(2010) and is based on the WRF Single-Moment 6-Class (WSM6) scheme. It predicts the 
mixing ratios of water vapor, cloud water, cloud ice, rain, snow and graupel as in the WSM6 
scheme, as well as the prognostic variables of cloud, rain water, and cloud condensation nuclei 
number concentrations. This scheme was added to the WRF model in 2009. 

1.3.2 Planetary Boundary Layer Schemes 

Planetary boundary layer (PBL) schemes play a critical role in simulating the boundary layer 
by parameterizing the vertical turbulent sub-grid scale fluxes in the PBL and the atmospheric 
column as a whole. The schemes determine the flux profiles in the boundary layer and stable 
layer above and provide vertical atmospheric tendencies of temperature, moisture, and 
horizontal momentum (Hu et al. 2010). Therefore, these PBL schemes are important for 
forecasting the initiation of convection and wind. A closure scheme is needed to determine the 
turbulent fluxes from the mean variables (Holt and Raman 1988, Hu et al. 2010). PBL schemes 
are generally divided into first-order or 1.5 order, also known as turbulent kinetic energy (TKE), 
closure schemes. First order closure schemes do not require additional prognostic equations to 
determine the effects of turbulence on the mean variables while 1.5 order (TKE) scheme 
requires an additional prognostic equation of the TKE (Shin and Hong 2011). The AMU tested 
three PBL algorithms in this task and they are described below: Mellor-Yamada-Janjic (MYJ), 
Yonsei University (YSU) and NCEP Global Forecast System (GFS). 

The MYJ PBL scheme is a TKE closure scheme that requires one additional prognostic 
equation of TKE and uses the Mellor-Yamada Level turbulence closure model to represent 
turbulence above the surface layer (Mellor and Yamada 1982 and Janjic 1990, 1996, 2002). It 
determines the eddy diffusion coefficients from prognostically calculated TKE. 

The YSU PBL scheme is a first-order closure scheme that is an update of the Medium- 
Range Forecast (MRF) PBL scheme (Hong and Pan 1996). It relies heavily on the Richardson 
number (Ri) to compute the PBL height for stable, unstable and neutral PBLs. The PBL height 
under this scheme is the point at which an Ri of 0.0 is reached. 
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The NCEP GFS PBL scheme is also a first-order vertical diffusion scheme that is closely 
related to the MRF PBL scheme. This scheme is used operationally at NCEP. More details can 
be found in Troen and Mahrt (1986) and Hong and Pan (1996). 

1.4 Report Format and Outline 

This report presents the findings from a study of model sensitivities for predicting unique 
weather phenomena at the ER and WFF. This analysis examined different dynamical cores, grid 
configurations, and physical schemes to determine the impact on model skill. The AMU 
assessed model skill by using an objective statistical analysis that included the mean error (ME) 
and the root mean square error (RMSE). Precipitation forecasts were evaluated using the 
Method for Object-Based Diagnostic Evaluation (MODE). Section 2 describes the model 
configuration, forecast validation technique, warm season and cool season results, and 
recommendations for the ER. Section 3 describes the model configuration, forecast validation 
technique, fall and spring season results, and recommendations for WFF. Each section is 
written to be a stand-alone guide for each range. 

2. Eastern Range 

This section describes the model configuration and test cases, results, and 
recommendations for an operational model for the ER. 

2.1 Model Configuration and Test Cases 

The most important part of this task was the model configuration. The ER wanted an 
operational model that could offer them information that is not readily available from national 
models. This included 

• a higher resolution model that could run at a more frequent interval than the national 
models, 

• a model that could output forecast data at a more frequent interval, 

• a model that could be configured for the unique weather patterns in east-central 
Florida, and 

• all of the above being feasible in an operational time frame. 

To meet these requirements, the AMU tested various WRF model configurations by varying the 
dynamical core, grid spacing, domain size, and forecast length. This enabled the AMU to 
determine the optimal configuration that allowed for the largest domain size and highest 
resolution necessary to capture unique weather phenomena on the ER with the shortest wall- 
clock run time. 

The AMU conducted testing on a 6-year old local modeling cluster that consists of 24 
nodes/48 processors. To estimate how the model would run on more modern equipment, the 
AMU scaled the model run-time information to that of a newer, faster cluster with similar 
specifications. The domain size or resolution may be further increased depending on the 
specifications of the cluster used to run the WRF model. In addition, the adaptive time-step 
option was turned on in the WRF model. The time-step length is a user-configurable parameter 
that normally remains constant throughout the model run. Choosing an appropriate time-step is 
important since one that is too long will cause instability and/or model failure and one that is too 
short is a waste of computing power. The adaptive time-step allows the time-step to be varied 
as necessary through the simulation to improve model performance and reduce the instability. 
Model run-time can be reduced using this option. 
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Choosing one model configuration that would best fit the requirements was not a linear 
process. The AMU first chose two preliminary domain configurations using the NMM core for 
testing. Initially, the NMM was chosen over the ARW due to the significantly faster run-time. 
After running these two configurations and validating the results, the AMU decided to compare 
an ARW configuration as well. Results from those tests indicated that the ARW configuration 
produced a better overall forecast than the two NMM configurations. Based on these findings, 
the AMU decided to run the ARW with different physics options and compare the results. At the 
end of the testing process, forecasts from the following 10 configurations 1 , two NMM and eight 
ARW, were compared: 

Configuration 1: NMM core, 3 km outer domain and 1 km inner domain, Ferrier microphysics 
scheme, MYJ PBL scheme (NMM 3/1), 

Configuration 2: NMM core, 2 km outer domain and 0.67 km inner domain, Ferrier microphysics 
scheme, MYJ PBL scheme (NMM 2/0.6), 

Configuration 3: ARW core, 2 km outer domain and 0.67 km inner domain, Lin microphysics 
scheme, YSU PBL scheme (Lin-YSU), 

Configuration 4: ARW core, 2 km outer domain and 0.67 km inner domain, Ferrier microphysics 
scheme, YSU PBL scheme (Ferrier-YSU), 

Configuration 5: ARW core, 2 km outer domain and 0.67 km inner domain, WDM6 microphysics 
scheme, YSU PBL scheme (WDM6-YSU), 

Configuration 6: ARW core, 2 km outer domain and 0.67 km inner domain, Goddard 
microphysics scheme, YSU PBL scheme (Goddard-YSU), 

Configuration 7: ARW core, 2 km outer domain and 0.67 km inner domain, Ferrier microphysics 
scheme, MYJ PBL scheme (Ferrier-MYJ), 

Configuration 8: ARW core, 2 km outer domain and 0.67 km inner domain, WDM6 microphysics 
scheme, MYJ PBL scheme (WDM6-MYJ), 

Configuration 9: ARW core, 2 km outer domain and 0.67 km inner domain, Ferrier microphysics 
scheme, NCEP GFS PBL scheme (Ferrier-GFS), 

Configuration 10: ARW core, 2 km outer domain and 0.67 km inner domain, WDM6 
microphysics scheme, NCEP GFS PBL scheme (WDM6-GFS). 

All other physics parameters were the same for each ARW model run. The NMM runs did 
not use the same physics as the ARW runs since physics options are more limited for the NMM. 
Table 1 lists the physics options used in both the ARW and NMM runs that were held constant. 
Note that cumulus parameterization schemes are generally not used with grid spacing that is 
less than 3 km. 


1 Model configurations will be referenced by their name in parentheses in the results sections 
(2.3 and 2.4). 
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Table 1. List of additional physics options used for each model run for both the ARW and 
NMM cores. 

Physics Option 

ARW 

NMM 

Cumulus 

Parameterization 

None 

None 

Land surface option 

Noah Land Surface Model 
(Chen and Dudhia 2001) 

Noah Land Surface Model 
(Chen and Dudhia 2001) 

Shortwave radiation 
scheme 

Dudhia (Dudhia 1989) 

GFDL (Lacis and Hansen 1974) 

Longwave radiation 
scheme 

RRTM (Mlaweret al. 1997) 

GFDL (Fels and Schwarzkopf 
1975; Schwarzkopf and Fels 
1985, 1991) 


Figure 1 shows domain size of both the inner and outer grid for each of the WRF 
configurations. The outer grid had the larger spacing (3 or 2 km) and the inner grid had smaller 
spacing (1 or 0.67 km) between grid points. Although the grid resolutions and dynamical core 
varied among the different configurations, the domain sizes for each configuration were roughly 
the same. 

The AMU ran each model simulation at the specified horizontal resolution with 45 irregularly 
spaced, vertical sigma levels up to 50 mb. For the warm season, each run was initiated at 1500 
UTC and integrated 9 hours once per day using the 12 km North American Mesoscale (NAM) 
model for boundary and initial conditions. Test cases from August 2011 were run to see if the 
model was able to capture the warm season convective initiation, onset of the sea breeze, and 
other warm season phenomena. Configurations 1-3 were run for the entire month of August 

2011. Configurations 4-10 were run from 1-7 August 2011 instead of the whole month due to 
time constraints. 

Configurations 1 , 3, and 4 were chosen to compare results in the cool season based on the 
results from the warm season model runs. Configurations 3 and 4 performed consistently well 
(see Section 2.3), while Configuration 1 was included to determine if the NMM performed better 
in the cool season. The AMU ran test cases for the three configurations for 18-25 February 

2012. Two 9-hour forecasts were initialized each day at 0000 and 1200 UTC using the 12 km 
NAM model for boundary and initial conditions, Short-term Prediction Research and Transition 
Center (SPoRT) Land Information System (LIS) land surface data, and SPoRT sea surface 
temperature (SST) data. The LIS and SST data were not used in the warm season comparison 
as the AMU did not have the archive data available. Two 9-hour forecasts were run in the cool 
season in order to capture a variety of phenomena including synoptic systems, frontal 
passages, and cool winter nights. 
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Figure 1. Map of east-central Florida showing the model domain boundaries. The red rectangle 
shows the outer domain and the yellow rectangle the inner domain for all configurations 
regardless of horizontal resolution or domain core. 


2.2 Forecast Validation 

The AMU validated the WRF model forecasts using simple statistics that compared locally 
available observational surface and upper air data to the forecast data. Precipitation forecasts 
were compared to nationally available rainfall data using a technique developed at NCAR. Two 
national models, the 13 km Rapid Update Cycle (RUC) and the 12 km NAM, were also validated 
against the same KSC/CCAFS wind towers in order to compare their performance to that of the 
WRF model surface forecasts. 

2.2.1 Observed Data 

Surface forecasts were validated with data from seven KSC/CCAFS wind towers, shown on 
the map in Figure 2. The model 10 m wind speed and direction and 2 m temperature (T) and 
dewpoint temperature (T d ) were validated with the 6 ft wind speed and direction and 54 ft T and 
T d , respectively, from towers 2, 6, 108, 110, and 512. Model 10 m wind speed and direction 
were validated with 30 ft wind speed and direction from towers 51 1 and 513. 
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Upper air forecasts valid at 2200 UTC (7-hour forecast) for the warm season were validated 
with the daily 2200 UTC CCAFS (XMR) sounding (Figure 2). The model wind speed and 
direction, T, and T d were compared to the observed values at 50 mb increments from 1000 mb 
to 100 mb. 

Forecast accumulated rainfall was compared to the NCEP Stage-ll 
(http://www.emc.ncep.noaa.gov/mmb/vlin/pcDanl/staae2/) analysis data to verify precipitation. 
This analysis is a national multi-sensor hourly precipitation analysis based on hourly radar 
precipitation estimates from the -140 Level II Weather Surveillance Radar-1988 Doppler (WSR- 
88D) radars in the continental United States and -3,000 automated gauge reports. The rainfall 
analysis provides hourly rainfall accumulation on a 4 km grid. 



Figure 2. The locations of the seven KSC/CCAFS wind towers and their numbers that were 
used to validate the WRF forecasts on the ER are shown in red. The XMR RAOB site used to 
validate the upper air forecasts is shown in yellow. 
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2.2.2 Verification Statistics 


The AMU calculated the same verification statistics for both the surface and upper air 
forecasts. The objective statistical analysis included the model bias, mean error (ME), and the 
root mean square error (RMSE). For the surface analysis, the observed wind speed and 
direction, T, and T d were compared to the WRF forecast values output every 30 minutes. 
Statistics that evaluated the WRF model against the two national models compared the 
observations to the output data every 3 hours, which corresponds to the availability of the NAM 
model output. All statistics were computed using data from the inner WRF grids. 

The model bias was calculated for each model forecast against every observation from each 
tower and the daily RAOB at each available level. This difference was calculated by subtracting 
the observed parameter from the model forecast as shown by the equation below. 

Diff WRF = Model Forecast - Tower/RAOB Observation 

In order to make sense of the bias data, the ME was calculated for all tower and RAOB data 
during the period of interest. The ME is a measure of the overall bias of the model parameter 
being compared. A perfect forecast has ME = 0. It is defined as 




(=1 


where: 


n = number of model output times and/or vertical levels over the forecast period, 


ft = WRF forecast of T, T d , wind speed, or wind direction (surface or upper air), and 

o, = observed T, T d , wind speed, or wind direction (surface or upper air). 

The model RMSE was calculated to measure the magnitude of the error. It is useful in 
determining whether the forecasts produced large errors, as it gives relatively high weight to 
large errors. It is calculated using the following equations: 


MSE 


n 


1 V- 



i = 1 


2 


RMSE = VMSE 


where n, f„ and o, are defined as above. 

To verify precipitation, rainfall accumulation throughout the entire forecast was compared to 
the observed rainfall over the same time period. MODE was used to determine the skill of each 
model configuration and it is available as part of the Model Evaluation Tools (MET) software that 
was developed by the NCAR Developmental Testbed Center 
(http://verif.rap.ucar.edu/eval/hwt/201 1 /spatial eval.php) . It is a state-of-the-art suite of 
verification tools that uses output from the WRF model to compute standard verification scores 
comparing gridded model data to point or gridded observations. 

MODE is an object-based verification that compares gridded observations to gridded 
forecasts. It resolves and compares objects, such as areas of accumulated rainfall, in both the 
forecast and observed fields. The objects can be described geometrically and then the attributes 
of the objects can be compared (Davis et al. 2006). MODE outputs statistics that describe the 
correlation between the objects and allows the user to identify forecast strengths or 
weaknesses. Details about how objects are identified and characterized can be found in Davis 
et al. (2006). For this report, the objects of interest are areas of accumulated rainfall. Therefore, 
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references to 'objects’ are references to areas of resolved accumulated rainfall throughout the 
forecast period. 

Once the objects have been identified, their various properties are evaluated and compared. 
The object attributes examined by the AMU in this task included the centroid distance, area 
ratio, and total interest value. The centroid distance is the vector difference between the 
centroids of the forecast and observed objects. It describes the location bias in the forecasts. 
The smaller the distance between the centroids, the better the forecast. The area ratio 
compares the area, or number of grid squares, the forecast object occupies compared to the 
observed object. An area ratio of 1 is considered a perfect correlation. Interest value is defined 
as the differences in particular attributes between the forecast and observed objects. Interest 
values of 0 indicate no interest, or a poor forecast, while a value of 1 indicates high interest, or a 
good forecast. The total interest value is a weighted sum of specific interest values and is used 
as an overall indicator of the quality of the precipitation forecast. Total interest value is large 
when forecast and observed objects are well correlated (are roughly the same size and are 
close to each other) and is small when they are not well correlated. 

2.3 Warm Season Results 

Surface, upper air, and precipitation forecasts were compared to observations during part of 
the 201 1 warm season. Different configurations of the WRF model were compared to determine 
if one configuration would consistently outperform the others. 

2.3.1 Surface Forecasts 

The AMU validated the WRF model forecasts with the KSC/CCAFS wind tower data. Two 
national models, the 13 km RUC and the 12 km NAM, were also validated against the same 
KSC/CCAFS wind towers for the month of August 2011 in order to compare their performance 
to that of the WRF model forecasts. Color-coded Good-Bad-Neutral (GBN) tables were 
populated with the computed ME and RMSE statistics. The tables are used to show which 
model configuration performed the best and worst. A good (green) rating indicates that the 
model configuration had the lowest ME or RMSE values among the different configurations. A 
bad (red) rating had the highest ME or RMSE values and neutral (yellow) fell in the middle. 

Table 2 shows the GBN chart of the average 30-day ME for August 201 1 between all seven 
wind towers and three WRF configurations and the two national models. The RUC model had 
the lowest ME for wind direction and T d while the ARW configuration (Lin-YSU) had the lowest 
ME for wind speed and temperature. The AMU anticipated that the national models would 
outperform the WRF configurations since 

• no observational data were assimilated into the WRF model, 

• 3-hourly NAM forecasts were used for the WRF initial boundary conditions versus a 
1 -hourly forecast from the RUC since soil moisture and temperature data were not 
available in the archived RUC forecasts, and 

• no high-resolution LIS or SST data were used since the AMU did not have the 
archive data. 

Therefore it was surprising and encouraging to see the ARW configuration (Lin-YSU) perform so 
well. The national models were included in the comparison mainly to determine if the WRF 
forecasts were reasonable. 
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Table 2. GBN chart of the average 30-day ME for August 2011 for all seven 
KSC/CCAFS wind towers for the three initial WRF configurations and two 
national models. 

30-day: Mean Error or Monthly Bias - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

12 km NAM 

42.7 

0.986 

-0.240 

- 1.000 

13 km RUC 

37.9 

0.814 

-1.400 

0.350 

NMM 3/1 

42.3 

1.629 

-0.660 

-1.174 

NMM 2/0.6 

44.4 

1.552 

-0.713 

-1.377 

Lin-YSU 

47.7 

0.463 

-0.236 

-2.788 


Of the three WRF forecasts run for August 2011, the only ARW configuration included in the 
initial comparison, Lin-YSU, produced the best wind speed and temperature forecasts, but the 
worst wind direction and T d forecasts (Table 3). NMM 3/1 performed best for wind direction and 
T d and worst for wind speed. NMM 2/0.6 fell somewhere in the middle for all forecasts except 
temperature. An RMSE value of 15.417 for temperature for NMM 2/0.6 indicates that there were 
large errors in some of the forecasts. 


Table 3. GBN chart of the average 30-day ME and RMSE for 1-30 August 
2011 for all seven KSC/CCAFS wind towers for the three initial WRF 
configurations. 

30-day: Mean Error or Monthly Bias - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

NMM 3/1 

42.3 

1.629 

-0.660 

-1.174 

NMM 2/0.6 

44.4 

1.552 

-0.713 

-1.377 

Lin-YSU 

47.7 

0.463 

-0.236 

-2.788 

30-day: RMSE - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

NMM 3/1 

58.4 

2.558 

4.617 

3.115 

NMM 2/0.6 

62.3 

2.525 

15.417 

3.205 

Lin-YSU 

65.5 

1.879 

3.943 

4.162 


Table 4 gives the total number of good, bad, and neutral forecasts for the combined 30-day 
ME and RMSE of the three WRF configurations at the seven wind tower locations. Results 
indicate that Lin-YSU had the most ‘good’ forecasts followed closely by NMM 3/1. 
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Table 4. GBN totals from the ME and RMSE values for the 
three initial WRF configuration forecasts at the seven 
KSC/CCAFS wind towers during 1-30 August 201 1 . 

1-30 August 2011 Totals 

Model 

Good 

Neutral 

Bad 

NMM 3/1 

21 

13 

11 

NMM 2/0.6 

3 

29 

14 

Lin-YSU 

22 

3 

23 


Based on these results and those of the precipitation and upper air comparison (detailed in 
Sections 2.3.2 and 2.3.3), the AMU decided to run seven more ARW configurations 
(Configurations 4-10) with different physics options, for a total of eight ARW runs, and compare 
the results. Due to time constraints, all configurations were run once per day at 1500 UTC from 
1-7 August 2011 instead of for the whole month. In both ARW runs that used the GFS PBL 
scheme, the 2 m shelter temperatures routinely increased to unrealistic values of over 100 °F. 
Therefore, both Ferrier-GFS and WDM6-GFS (Configurations 9 and 10, respectively) were 
dropped from the comparison. Based on the 7-day ME for all towers (Table 5), both Lin-YSU 
and Ferrier-YSU outperformed the rest of the ARW model runs. Examining Table 6, Lin-YSU 
had the most ‘good’ forecasts and the least ‘bad’ forecasts, followed by Ferrier-YSU. 


Table 5. GBN chart of the average ME and RMSE for 1-7 August 2011 for all 
seven KSC/CCAFS wind towers for the six ARW configurations. 

7-day: Mean Error - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Lin-YSU 

47.3 

-0.086 

-0.900 

-1.969 

Ferrier-YSU 

46.2 

0.069 

-1.064 

-1.844 

WDM6-YSU 

50.3 

0.009 

-1.054 

-2.323 

Goddard-YSU 

47.6 

-0.023 

-1.077 

-1.952 

Ferrier-MYJ 

47.7 

1.276 

-0.955 

-1.575 

WDM6-MYJ 

48.5 

0.987 

-0.597 

-1.836 

7-day: RMSE - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Lin-YSU 

63.1 

1.400 

2.369 

3.224 

Ferrier-YSU 

62.6 

1.403 

2.432 

3.010 

WDM6-YSU 

67.1 

1.585 

2.563 

3.769 

Goddard-YSU 

63.6 

1.497 

2.364 

3.117 

Ferrier-MYJ 

64.1 

2.175 

2.835 

3.096 

WDM6-MYJ 

65.5 

1.956 

2.571 

3.567 
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Table 6. GBN totals from the ME and RMSE values for the six 
ARW configuration forecasts at the seven KSC/CCAFS wind 

towers during 1-7 August 201 1 . 



1-7 August 2011 Totals 

Model 

Good 

Neutral 

Bad 

Lin-YSU 

13 

32 

1 

Ferrier-YSU 

12 

32 

2 

WDM6-YSU 

4 

26 

16 

Goddard-YSU 

4 

41 

1 

Ferrier-MYJ 

6 

22 

18 

WDM6-MYJ 

7 

31 

8 


2.3.2 Upper Air Forecasts 

The AMU validated the six ARW configurations against the daily 2200 UTC XMR sounding. 
Additional XMR sounding data are available at 1500 UTC in the warm season, however these 
data were not used in the comparison since the WRF forecasts were initialized at 1500 UTC. All 
model configurations had a similar 0-hour forecast and, therefore, the ME statistics between the 
observed and forecast values were essentially the same. The XMR sounding data was 
interpolated to match the model output that was available every 50 mb from 1000-100 mb. The 
average 7-day ME and RMSE were computed for wind direction, wind speed, T, and T d 
(Table 7). Based on the results, Lin-YSU produced the best upper-air forecast followed by 
Ferrier-YSU. 


Table 7. GBN chart of the average ME and RMSE for 1-7 August 201 1 for the 
2200 UTC XMR RAOB for the six ARW configurations. 

7-day: Mean Error - XMR RAOB 

Model 

Wind Dir 

(deg) 

WindSpd 

(m/s) 

Temp (F) 

Dewpt (F) 

Lin-YSU 

35.0 

-0.146 

0.436 

-5.503 

Ferrier-YSU 

38.5 

-0.046 

0.387 

-4.826 

WDM6-YSU 

37.7 

0.159 

0.491 

-5.584 

Goddard-YSU 

35.9 

-0.170 

0.458 

-5.631 

Ferrier-MYJ 

39.1 

0.044 

0.466 

-5.345 

WDM6-MYJ 

35.1 

-0.102 

0.539 

-6.761 

7-day: RMSE - XMR RAOB 

Model 

Wind Dir 

(deg) 

WindSpd 

(m/s) 

Temp (F) 

Dewpt (F) 

Lin-YSU 

50.2 

2.542 

1.785 

15.381 

Ferrier-YSU 

54.5 

2.545 

1.818 

15.970 

WDM6-YSU 

53.3 

2.734 

1.773 

15.797 

Goddard-YSU 

50.5 

2.611 

1.826 

15.591 

Ferrier-MYJ 

55.5 

2.626 

1.891 

15.994 

WDM6-MYJ 

49.8 

2.565 

1.841 

15.902 
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2.3.3 Precipitation Forecasts 

The AMU compared precipitation forecasts from NMM 3/1, NMM 2/0.6, and Lin-YSU to 
determine performance differences between the NMM and ARW. The 9-hour forecast 
accumulated rainfall was compared to the 9-hour accumulation of observed rain using the 
NCEP Stage-ll analysis data for each day from 1-30 August 2011. The monthly summary 
statistics of centroid distance, area ratio, and interest function from the MODE software are 
shown in Table 8. The centroid distance (km) results indicate that the Lin-YSU (ARW) 
precipitation matched the location of the observed precipitation most closely. An interest value 
of 0.92 indicates that overall the forecast correlates well with the observed precipitation. The 
area ratio for NMM 3/1 indicates that the forecast most closely matches the areal coverage of 
observed precipitation, followed by Lin-YSU. Overall, the ARW configuration outperformed both 
NMM configurations. 


Table 8. GBN chart for the average 30-day MODE statistics for 1-30 August 
201 1 over the ER WRF domain for the initial three WRF configurations. 

30-day: MODE - Precipitation Statistics 

Model 

Centroid Distance (km) 

Area Ratio 

Interest 

NMM 3/1 

22.43 

0.54 

0.91 

NMM 2/0.6 

22.68 

0.33 

0.85 

Lin-YSU 

14.33 

0.52 

0.92 


Based on these results, the same summary statistics were computed for the six additional 
ARW configurations for 1-7 August 2011 (Table 9). It is interesting to note that Ferrier-YSU 
produced the worst precipitation forecasts, but some of the best surface and upper air forecasts. 
In general, results indicate that the WDM6 microphysics option (WDM6-YSU and WDM6-MYJ) 
tended to produce the best precipitation forecasts. It should be noted that it took some time for 
precipitation to spin up in all model runs. Once observational data is assimilated, spin up time 
will decrease and individual performance of any model configuration may improve. 


Table 9. GBN chart for the average MODE statistics for 1-7 August 2011 over 
the ER WRF domain for the six ARW configurations. 

7-day: MODE - Precipitation Statistics 

Model 

Centroid Distance (km) 

Area Ratio 

Interest 

Lin-YSU 

9.67 

0.69 

0.98 

Ferrier-YSU 

19.10 

0.43 

0.92 

WDM6-YSU 

16.36 

0.79 

0.98 

Goddard-YSU 

13.45 

0.73 

0.98 

Ferrier-MYJ 

9.60 

0.58 

0.97 

WDM6-MYJ 

13.45 

0.72 

0.99 


2.4 Cool Season Results 

Based on the warm season results, the AMU chose to compare the NMM 3/1, Lin-YSU, and 
Ferrier-YSU in the cool season. Both Lin-YSU and Ferrier-YSU performed consistently well, 
while NMM 3/1 was included as the better of the two NMM configurations (Table 8) to determine 
if the NMM performed better in the cool season. Test cases for the three configurations were 
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run for eight days from 18-25 February 2012. Two 9-hour forecasts were produced each day 
starting at 0000 and 1200 UTC using the 12 km NAM model for boundary and initial conditions, 
SPoRT LIS surface data, and SPoRT SST data. 

2.4.1 Surface Variables 

The AMU validated the WRF model forecasts with the KSC/CCAFS wind tower data from 
towers 2, 6, 108, and 110 as shown in Figure 2. Data for towers 511, 512, and 513 were not 
available between 18-25 February 2012. The two national models were also validated against 
the same KSC/CCAFS wind towers in order to compare their performance to that of the three 
WRF model forecasts. As in the warm season, Table 10 indicates that Lin-YSU had as many 
good forecasts for the ME and RMSE as the 13 km RUC. NMM 3/1 and the 12 km NAM 
performed the worst. In general, the cool season ME for wind direction, temperature, and 
dewpoint temperature were smaller than that for the warm season. Table 11 shows the 8-day 
GBN totals for ME and RMSE for all configurations and indicates that Lin-YSU had the most 
‘good’ forecasts and outperformed all other configurations, including the 13 km RUC. 


Table 10. GBN chart of the average ME and RMSE for 18-25 February 2012 
for four KSC/CCAFS wind towers for the three initial WRF configurations and 
two national models. 

8-day: Mean Error - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

NAM 

29.5 

0.409 

0.487 

0.875 

RUC 

29.0 

0.890 

0.338 

1.387 

NMM 3/1 

29.5 

1.154 

0.764 

0.693 

Lin-YSU 

29.7 

-0.047 

0.816 

0.041 

Ferrier-YSU 

30.3 

-0.039 

0.769 

0.053 

8-day: RMSE - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

NAM 

49.7 

1.788 

3.014 

1.823 

RUC 

45.5 

1.973 

4.055 

2.443 

NMM 3/1 

49.1 

2.335 

3.220 

1.809 

Lin-YSU 

47.6 

1.664 

2.734 

1.516 

Ferrier-YSU 

49.4 

1.702 

2.751 

1.536 
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Table 1 1 . GBN totals from the ME and RMSE values for the three initial WRF 
configurations and two national models at four KSC/CCAFS wind towers 

during 18-25 February 2012. 



18-25 February 2012 Totals 

Model 

Good 

Neutral 

Bad 

NAM 

6 

21 

3 

RUC 

8 

11 

11 

NMM 3/1 

2 

17 

11 

Lin-YSU 

11 

15 

4 

Ferrier-YSU 

3 

26 

1 


2.4.2 Upper Air forecasts 

XMR data were available at 1100 and 2300 UTC during the cool season. Since these times 
did not occur within the 9-hour 0000 or 1200 UTC forecast periods, the AMU could not compare 
the model output to the sounding data. 

2.4.3 Precipitation forecasts 

The AMU compared the 9-hour accumulation of observed rain using the NCEP Stage-ll 
analysis data to the 9-hour forecast accumulated rainfall from the 0000 and 1200 UTC model 
runs for 18-25 February 2012. The summary statistics of centroid distance (km), area ratio, and 
interest function from the MODE software are shown in Table 12. The ‘NA’ designations for 
NMM 3/1 indicate that the model failed to forecast precipitation that matched the observations. 
The results indicate that Lin-YSU produced the best precipitation forecast, although it did not 
perform much better than Ferrier-YSU. The model did a better job of forecasting the warm 
season precipitation vs. the cold season precipitation. This can be explained by having less 
precipitation events in the cool season period of record and less rainfall during those events. In 
most cases the model did predict precipitation when it was observed, but forecast less 
precipitation than the observed events. In these cases, the forecast precipitation was filtered out 
due to the methods used by the MODE tool to resolve forecast and observed precipitation 
objects. The precipitation thresholds can be varied and were adjusted to account for the smaller 
rainfall amounts. However, even with the smaller thresholds the forecast precipitation was too 
light and was filtered out. 


Table 12. GBN chart for the average MODE statistics for 18-25 February 2012 

over the ER WRF domain. 



8-Day: MODE - Precipitation Statistics 

Model 

Centroid Distance 

Area Ratio 

Interest 

NMM 3/1 

NA 

NA 

NA 

Lin-YSU 

20.97 

0.39 

0.80 

Ferrier-YSU 

20.96 

0.36 

0.79 
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2.5 Additional Comparisons 

The AMU compared three Lin-YSU configurations run once per day at 1500 UTC from 1-7 
August 2011 with varying horizontal grid spacing to determine if a slightly coarser or finer 
resolution would have much impact on the results. The first configuration had a 2 km outer 
domain and a 0.67 km inner domain (Lin-YSU 2/0.6), the second had a 3 km outer and 1 km 
inner domain (Lin-YSU 3/1), and the third had a 1 km outer and 0.3 km inner domain (Lin-YSU 
1/0.3). The ME and RMSE for wind direction, wind speed, T, and T d were computed for each 
configuration at the seven KSC/CCAFS wind towers. Results are shown in Table 13. The results 
indicate that the two higher resolution model configurations slightly outperformed the coarser 
resolution for wind speed, T, and T d . The Lin-YSU 3/1 outperformed the both the 2/0.67 and 
1/0.3 configurations for wind direction. 


Table 13. GBN chart of the average ME and RMSE for 1-7 August 2011for 
seven KSC/CCAFS wind towers for three ARW configurations with varying 
horizontal resolutions. 

7-day: Mean Error - All Towers 

Model 

Wind Dir 

(deg) 

WindSpd 

(m/s) 

Temp (F) 

Dewpt (F) 

Lin-YSU (2/0.6) 

47.4 

-0.086 

-0.880 

-1.950 

Lin-YSU (3/1) 

43.0 

0.143 

-2.240 

-1.950 

Lin-YSU (1/0.3) 

49.6 

-0.097 

-0.942 

-1.879 

7 day: RMSE - All Towers 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Lin-YSU (2/0.6) 

63.1 

1.400 

2.369 

3.224 

Lin-YSU (3/1) 

62.1 

1.534 

3.161 

3.353 

Lin-YSU (1/0.3) 

65.9 

1.335 

2.632 

3.039 


2.6 Recommendations and Future Work 

Warm and cool season results indicate that Lin-YSU is the optimal model configuration for 
the ER. It consistently produced the best surface and upper air forecasts, while performing fairly 
well for the precipitation forecasts. Therefore the AMU recommends running the WRF ARW 
over the ER using the Lin microphysical scheme and the YSU PBL scheme. 

Running the model at a high horizontal resolution will require either a decrease in domain 
size or an increase in model run time. The user must ultimately decide if the higher resolution is 
worth the cost of the increase in computing resources or a decrease in domain size. The AMU 
recommends running the model with a 2 km outer and 0.67 km inner domain. 

The next step in this study will be to provide a recommended local data assimilation (DA) 
and numerical forecast model design optimized for the ER to support space launch activities. 
The AMU will determine the best software and type of assimilation to use, as well as determine 
the best grid resolution for the initialization based on spatial and temporal availability of data and 
the wall clock run-time of the initialization. The AMU will also transition from the WRF EMS to 
the NASA Unified-WRF (NU-WRF), a NASA-specific version of the WRF that takes advantage 
of unique NASA software and datasets. 
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3. Wallops Flight Facility 

This section describes the model configuration and test cases, results, and 
recommendations for an operational model for WFF. 

3.1 Model Configuration and Test Cases 

The most important part of this task was the model configuration. The users at WFF could 
benefit from an operational model that could offer them information that is not readily available 
from national models. This includes 

• a higher resolution model that could run at a more frequent interval than the national 
models, 

• a model that could output forecast data at a more frequent interval, and 

• a model that could be configured for the unique weather patterns over the Delaware- 
Maryland-Virginia (Delmarva) Peninsula. 

To meet these requirements, the AMU tested various WRF model configurations by varying the 
dynamical core, grid spacing, and domain size while running a 24-hour forecast. This enabled 
the AMU to present different grid configurations to WFF to allow them to choose the optimal 
configuration for their forecast needs. 

The AMU conducted testing on a 6-year old local modeling cluster that consists of 24 
nodes/48 processors. To estimate how the model would run on more modern equipment, the 
AMU scaled the model run-time information to that of a newer, faster cluster with similar 
specifications. The domain size or resolution may be further increased depending on the 
specifications of the cluster used to run the WRF model. In addition, the adaptive time-step 
option was turned on in the WRF model. The time-step length is a user-configurable parameter 
that normally remains constant throughout the model run. Choosing an appropriate time-step is 
important since one that is too long will cause instability and/or model failure and one that is too 
short is a waste of computing power. The adaptive time-step allows the time-step to be varied 
as necessary through the simulation to improve model performance and reduce the instability. 
Model run-time can be reduced using this option. 

Personnel at WFF indicated that the optimal model configuration would allow for a 24-hour 
forecast and would include the entire Delmarva Peninsula. The AMU presented WFF with four 
configurations that varied in model core and domain size. WFF personnel chose a WRF ARW 
configuration with a 4 km outer and 1.33 km inner domain (Figure 3). The AMU decided to 
include an NMM configuration to test whether it produced a better forecast than the ARW 
(Figure 4). In addition, the AMU decided to run the ARW with different physics options and 
compare the results. The physics options were chosen based on the findings from the ER 
portion of the task (Section 2). Forecasts from the following four configurations 2 , three ARW and 
one NMM, were compared: 

Configuration 1: ARW core, 4 km outer domain and 1.33 km inner domain, Ferrier microphysics 
scheme, YSU PBL scheme (Ferrier-YSU), 

Configuration 2: ARW core, 4 km outer domain and 1.33 km inner domain, Lin microphysics 
scheme, YSU PBL scheme (Lin-YSU), 

Configuration 3: ARW core, 4 km outer domain and 1.33 km inner domain, WDM6 microphysics 
scheme, YSU PBL scheme (WDM6-YSU), 


2 Model configurations will be referenced by their name in parentheses in the results sections 
(3.3 and 3.4). 
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Configuration 4: NMM core, 3 km outer domain and 1 km inner domain, Ferrier microphysics 
scheme, MYJ PBL scheme (NMM 3/1). 


All other physics parameters were the same for each ARW model run. The NMM runs did 
not use the same physics as the ARW runs since physics options are more limited for the NMM. 
Table 14 lists the physics options used in both the ARW and NMM runs that were held constant. 



Figure 3. Map of the Delmarva Peninsula showing the WRF ARW model 
domain boundaries. The red rectangle shows the outer 4 km domain and the 
yellow rectangle the inner 1.33 km domain. 
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Figure 4. Map of the Delmarva Peninsula showing the WRF NMM model domain 
boundaries. The red rectangle shows the outer 3 km domain and the yellow 
rectangle the inner 1 km domain. 
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Table 14. List of additional physics options used for each model run for both the ARW and 
NMM cores. 

Physics Option 

ARW 

NMM 

Cumulus 

Parameterization 

None 

None 1 

Land surface option 

Noah Land Surface Model 
(Chen and Dudhia 2001) 

Noah Land Surface Model 
(Chen and Dudhia 2001) 

Shortwave radiation 
scheme 

Dudhia (Dudhia 1989) 

GFDL (Lads and Hansen 1974) 

Longwave radiation 
scheme 

RRTM (Mlawer et al. 1997) 

GFDL (Fels and Schwarzkopf 
1975; Schwarzkopf and Fels 
1985, 1991) 


The AMU ran each model simulation at the specified horizontal resolution with 45 irregularly 
spaced, vertical sigma levels. Each run was integrated 24 hours twice per day and initialized at 
0000 and 1200 UTC using the 12 km NAM model for boundary and initial conditions, SPoRT 
LIS surface data, and SPoRT SST data. Test cases for the ARW configurations were run from 
1-14 November 201 1 and 1-30 April 2012 to see if the model was able to accurately forecast the 
spring and fall season precipitation, temperature, and winds. An abbreviated set of test cases 
were run for the NMM configuration since the AMU had determined that the ARW performed 
better than the NMM in the ER portion of the task (Section 2). Test cases for the NMM were run 
from 1-7 November 2011. 

3.2 Forecast Validation 

The AMU validated the WRF model forecasts using simple statistics that compared locally 
available observational surface and upper air data to the forecast data. Precipitation forecasts 
were compared to nationally available rainfall data using a technique developed at NCAR. The 
12 km NAM was also validated against the same observational data in order to compare its 
performance to that of the WRF model surface forecasts. The 13 km RUC was not evaluated 
because archive RUC forecast data is only available out to 12 hours. 

3.2.1 Observed Data 

Surface forecasts from the ARW runs were validated with data from seven METAR stations 
and four weather buoys located within the domain of interest. Figure 5 shows the seven METAR 
sites in red and four buoy locations in green. The model 10 m wind speed and direction and 2 m 
T and T d were validated with the observed surface wind speed and direction, T, and T d . For 
comparisons that involved the NMM model, surface forecasts were validated with data from 
three METAR sites (KWAL, KOXB, and KSBY) since those sites were the only ones that fell 
within the NMM inner domain. 

Upper air forecasts for the 0000 UTC initialized runs valid at 1200 and 0000 UTC (12- and 
24-hour forecasts) were validated with the daily 1200 and 0000 UTC sounding taken at KWAL 
(Figure 5). Upper air forecasts for the 1200 UTC initialized runs valid at 0000 and 1200 UTC 
(12- and 24-hour forecasts) were validated with the daily 0000 and 1200 UTC KWAL sounding. 
The model wind speed and direction, T, and T d were compared to the observed values at 10 
vertical levels: 1000, 850, 700, 500, 400, 300, 250, 200, 150, and 100 mb. 
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Forecast accumulated rainfall was compared to the NCEP Stage-1 1 analysis data to verify 
precipitation. This analysis is a national multi-sensor hourly precipitation analysis based on 
hourly radar precipitation estimates from the -140 WSR-88D radars in the continental United 
States and -3,000 automated gauge reports. The rainfall analysis provides hourly rainfall 
accumulation on a 4 km grid. 



Figure 5. Seven METAR (red) and four buoy (green) weather reporting stations that were used 
to validate the WRF forecasts for WFF. The sounding was taken at the KWAL location. 


3.2.2 Verification Statistics 

The AMU calculated the same verification statistics for both the surface an upper air 
forecasts. The objective statistical analysis included the model bias, ME, and the RMSE. For the 
surface analysis, the observed wind speed and direction, T, and T d were compared to the WRF 
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forecast values output every 60 minutes. Statistics that evaluated the WRF model against the 
two national models compared the observations to the output data every 3 hours, which 
corresponds to the availability of the NAM model output. All statistics were computed using data 
from the inner WRF grids. 

The model bias was calculated for each model forecast against every observation from each 
METAR and buoy report, and the daily RAOB at each available level. This difference was 
calculated by subtracting the observed parameter from the model forecast as shown by the 
equation below. 

DiffwRF = Model Forecast - METAR/RAOB Observation 


In order to make sense of the bias data, the ME was calculated for each METAR, buoy and 
the RAOB observation for the period of interest. The ME is a measure of the overall bias of the 
model parameter being compared. A perfect forecast has ME = 0. It is defined as: 


n i 


where: 


n = number of model output times and/or vertical levels over the forecast period, 


fi = WRF forecast of T, T d , wind speed, or wind direction (surface or upper air), and 

o, = observed T, T d , wind speed, or wind direction (surface or upper air). 

The model RMSE was calculated to measure the magnitude of the error. It is useful in 
determining whether the forecasts produced large errors, as it gives relatively high weight to 
large errors. It is calculated using the following equations: 


MSE 



RMSE = VMS£ 


where n, and o, are defined as above. 

To verify precipitation, rainfall accumulation throughout the entire forecast was compared to 
the observed rainfall over the same time period. MODE was used to determine the skill of each 
model configuration. MODE is available as part of the MET software that was developed by the 
NCAR Developmental Testbed Center (http://verif.rap.ucar.edu/eval/hwt/201 1 /spatial eval.php) . 
It is a state-of-the-art suite of verification tools that uses output from the WRF model to compute 
standard verification scores comparing gridded model data to point or gridded observations. 

MODE is an object-based verification that compares gridded observations to gridded 
forecasts. It resolves and compares objects, such as areas of accumulated rainfall, in both the 
forecast and observed fields. The objects can be described geometrically and then the attributes 
of the objects can be compared (Davis et al. 2005). MODE outputs statistics that describe the 
correlation between the objects and allows the user to identify forecast strengths or 
weaknesses. Details about how objects are identified and characterized can be found in Davis 
et al. (2005). For this report, the objects of interest are areas of accumulated rainfall. Therefore, 
references to ‘objects’ are references to areas of resolved accumulated rainfall throughout the 
forecast period. 

Once the objects have been identified, their various properties are evaluated and compared. 
The object attributes examined by the AMU in this task included the centroid distance, area 
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ratio, and total interest value. The centroid distance is the vector difference between the 
centroids of the forecast and observed objects. It describes the location bias in the forecasts. 
The smaller the distance between the centroids, the better the forecast. The area ratio 
compares the area, or number of grid squares, the forecast object occupies compared to the 
observed object. An area ratio of 1 is considered a perfect correlation. Interest value is defined 
as the differences in particular attributes between the forecast and observed objects. Interest 
values of 0 indicate no interest, or a poor forecast, while a value of 1 indicates high interest, or a 
good forecast. The total interest value is a weighted sum of specific interest values and is used 
as an overall indicator of the quality of the precipitation forecast. Total interest value is large 
when forecast and observed objects are well correlated (are roughly the same size and are 
close to each other) and is small when they are not well correlated. 

3.3 Fall Season Results 

The AMU compared the surface, upper air, and precipitation forecasts to observations 
during part of the 2011 fall season. Different configurations of the WRF model were compared 
to determine if one configuration would consistently outperform the others. Test cases involving 
the national model and the NMM configuration were run for seven days from 1-7 November 
2011. Test cases that compared the ARW configurations were run for 14 days from 1-14 
November 201 1 . The AMU produced two 24-hour forecasts each day starting at 0000 and 1200 
UTC using the 12 km NAM model for boundary and initial conditions, SPoRT LIS surface data, 
and SPoRT SST data. 

3.3.1 Surface Forecasts 

The AMU validated the WRF model forecasts with the local METAR data. The 12 km NAM 
was also validated against the same METAR data for the first week of November 201 1 in order 
to compare its performance to that of the WRF model forecasts. Color-coded GBN tables were 
populated with the computed ME and RMSE statistics. The tables are used to show which 
model configuration performed the best and worst. A good (green) rating indicates that the 
model configuration had the lowest ME or RMSE values among the different configurations. A 
bad (red) rating had the highest ME or RMSE values and neutral (yellow) fell in the middle. 

Table 15 shows the GBN chart of the average 7-day ME and RMSE for 1-7 November 2011 
between three METAR sites and four WRF configurations and one national model. The results 
were varied. The good forecasts outperformed the neutral forecasts only marginally. However, 
the bad forecasts were considerably worse than the good/neutral forecasts. In order to compare 
the forecasts it may be prudent to look at the lowest number of bad forecasts rather than 
focusing on the highest number of good forecasts. In this case, NMM 3/1 performed the worst 
followed by Lin-YSU and WDM6-YSU. The 12 km NAM fell somewhere in the middle while 
Ferrier-YSU marginally outperformed the rest of the configurations. The AMU anticipated that 
the national model would outperform the WRF configurations since no observational data was 
assimilated into the WRF model. Therefore it was surprising and encouraging to see the WRF 
configurations perform so well. The national model was included in the comparison mainly to 
determine if the WRF forecasts were reasonable. 

Table 16 gives the total number of good, bad, and neutral forecasts for the 7-day combined 
ME and RMSE of the three METAR sites for all WRF configurations. Focusing on minimizing the 
bad forecasts, Ferrier-YSU produced the least bad forecasts. The NAM model and NMM 3/1 
produced the most ‘good’ forecasts, but also had a fairly large number of ‘bad’ forecasts. 
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Table 15. GBN chart of the average 7-day ME and RMSE for 1-7 November 
201 1 for three METAR sites for four WRF configurations and one national 
model. 

7 day: Mean Error - 3 METAR Sites 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

30.54 

1.076 

-1.660 

-0.819 

Lin-YSU 

30.44 

1.039 

-1.791 

-0.857 

WDM6-YSU 

31.19 

1.104 

-1.592 

-0.791 

NMM 3/1 

24.78 

2.013 

2.545 

1.773 

12 km NAM 

26.98 

1.263 

1.954 

2.265 

7 day: RMSE - 3 METAR Sites 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

44.34 

1.976 

4.113 

3.111 

Lin-YSU 

44.67 

2.036 

4.220 

3.017 

WDM6-YSU 

44.54 

2.062 

4.071 

3.121 

NMM 3/1 

35.70 

2.818 

3.893 

2.812 

12 km NAM 

39.47 

2.017 

3.312 

3.100 


Table 16. GBN totals from the ME and RMSE values for the 

four WRF and one national model forecasts 
METAR sites during 1-7 November 201 1 . 

at the three 

1-7 November 2011 Totals 

Model 

Good 

Neutral 

Bad 

Ferrier-YSU 

3 

20 

1 

Lin-YSU 

2 

18 

4 ! 

WDM6-YSU 

4 

15 

5 

NMM 3/1 

7 

7 

9 

12 km NAM 

8 

11 

5 


The AMU ran an additional seven days in November 2011 for each of the three ARW 
configurations and compared the results from 1-14 November 2011. Table 17 shows the GBN 
chart of the average 14-day ME and RMSE for 1-14 November 2011 between all seven METAR 
and four buoy sites and the three ARW configurations. The chart indicates that Lin-YSU 
performed the best, however the difference in ME and RMSE values between all configurations 
is fairly negligible. Examining Table 18, Ferrier-YSU had the most ‘good’ forecasts and the least 
‘bad’ forecasts, while Lin-YSU had the most ‘bad’ forecasts. These results seem to conflict with 
those from Table 17, but can be explained by the negligible differences in forecast statistics for 
all ARW runs. 
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Table 17. GBN chart of the average ME and RMSE for 1-14 November 2011 
for all seven METAR and four buoy sites for three ARW configurations. 

14-day: Mean Error - All METAR/Buoy Sites 

Model 

Wind Dir 

(deg) 

WindSpd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

27.9 

0.778 

-2.651 

-0.631 

Lin-YSU 

27.6 

0.738 

-2.631 

-0.666 

WDM6-YSU 

28.9 

0.761 

-2.503 

-0.681 

14-day: RMSE - All METAR/Buoy Sites 

Model 

Wind Dir 

(deg) 

WindSpd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

40.1 

2.122 

4.354 

3.073 

Lin-YSU 

39.8 

2.168 

4.542 

3.173 

WDM6-YSU 

42.1 

2.127 

4.360 

3.044 


Table 18. GBN totals from the ME and RMSE values for the 
three ARW forecasts at all seven METAR and four buoy sites 
during 1-14 November 2011. 

1-14 November 2011 Totals 


Model 

Good 

Neutral 

Bad 

Ferrier-YSU 

26 

15 

15 

Lin-YSU 

18 

13 

25 

WDM6-YSU 

12 

28 

16 


3.3.2 Upper Air Forecasts 

The AMU validated the three ARW configurations against the daily 0000 and 1200 UTC 
KWAL sounding. Twelve and 24-hour upper air forecasts for the 0000 and 1200 UTC initialized 
runs were validated with the corresponding KWAL sounding. The AMU used 10 vertical levels in 
the KWAL sounding for validation: 1000, 850, 700, 500, 400, 300, 250, 200, 150, and 100 mb. 
The average 14-day ME and RMSE were computed for wind direction, wind speed, T, and T d 
(Table 19). Based on the results, WDM6-YSU produced the best upper-air forecast followed by 
Lin-YSU. Again, differences in the computed statistics were small with the exception of T d . 
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Table 19. GBN chart of the average ME and RMSE for 1-14 November 2011 
for the 0000 and 1200 UTC KWAL RAOB and the three ARW configurations. 

14-day: Mean Error - KWAL ROAB 

Model 

Wind Dir 

(deg) 

WindSpd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

10.87 

-0.866 

-0.076 

2.808 

Lin-YSU 

10.73 

-0.869 

-0.063 

-0.015 

WDM6-YSU 

10.62 

-0.861 

-0.083 

2.812 

14-day: RMSE - KWAL RAOB 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

18.91 

2.558 

2.105 

9.788 

Lin-YSU 

18.71 

2.549 

2.106 

0.744 

WDM6-YSU 

18.52 

2.533 

2.081 

9.796 


3.3.3 Precipitation Forecasts 

The AMU compared precipitation forecasts from the three ARW configurations to determine 
which produced the best forecast. The 24-hour forecast accumulated rainfall was compared to 
the 24-hour accumulation of observed rain using the NCEP Stage-ll analysis data for each day 
from 1-14 November 2011. The summary statistics of centroid distance, area ratio, and interest 
function from the MODE software are shown in Table 20. The centroid distance (km) results 
indicate that the Lin-YSU precipitation matched the location of the observed precipitation most 
closely. An interest value of 0.99 indicates that overall the Lin-YSU forecast correlates well with 
the observed precipitation. The area ratio for WDM6-YSU indicates that the forecast most 
closely matches the areal coverage of observed precipitation, followed by Ferrier-YSU. Lin-YSU 
performed worst in areal coverage of precipitation, but overall it outperformed the other 
configurations. It should be noted that during the 14 days there were not many precipitation 
events and there was little rainfall during those events. In most cases either the observed or 
forecast precipitation was filtered out due to the methods used by the MODE tool to resolve 
forecast and observed precipitation objects. The precipitation thresholds can be varied and were 
adjusted to account for the smaller rainfall amounts. However, even with the smaller thresholds 
the forecast precipitation was too light and was filtered out. In addition, once observational data 
is assimilated, individual performance of any model configuration may improve. 


Table 20. GBN chart for the average 14-day MODE statistics for 1- 

14 November 

201 1 over the WFF WRF domain for the three ARW configurations. 


14-day: MODE - Precipitation Statistics 

Model 

Centroid Distance (km) 

Area Ratio 

Interest 

Ferrier-YSU 

8.59 

0.97 

0.98 

Lin-YSU 

3.02 

0.88 

0.99 

WDM6-YSU 

8.82 

0.98 

0.98 


3.4 Spring Season Results 

Based on the fall season results, the AMU chose to compare the ARW configurations in the 
spring season. The NMM configuration was dropped from the comparison since it was only 
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included to determine if it produced a superior forecast, which it did not. Test cases for the three 
configurations were run for 30 days from 1-30 April 2012. Two 24-hour forecasts were produced 
each day starting at 0000 and 1200 UTC using the 12 km NAM model for boundary and initial 
conditions, SPoRT LIS surface data, and SPoRT SST data. 

3.4.1 Surface Variables 

The AMU validated the WRF model forecasts with the local METAR and buoy data for the 
month of April 2012. Table 21 shows the GBN chart of the average 30-day ME and RMSE for all 
three configurations. It indicates that differences in statistics for all configurations were 
negligible as in the fall season. Lin-YSU performed the best, followed by Ferrier-YSU. Table 22 
shows the total number of good, bad, and neutral forecasts for the 30-day combined ME and 
RMSE at the METAR and buoy sites. The results are consistent and show Lin-YSU performing 
the best followed by Ferrier-YSU. WDM6-YSU consistently performed the worst. 


Table 21. GBN chart of the average 30-day ME for 1-30 April 2012 for all 
seven METAR and four buoy sites for three ARW configurations. 

30-day: Mean Error or Monthly Bias - All METAR/Buoy Sites 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

26.68 

1.081 

-0.222 

-0.043 

Lin-YSU 

25.9 

1.123 

-0.101 

1.575 

WDM6-YSU 

26.70 

1.099 

-0.148 

0.013 

30-day: RMSE - All METAR/Buoy Sites 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

38.9 

2.2746 

2.260 

1.539 

Lin-YSU 

37.5 

2.2753 

1.967 

1.527 

WDM6-YSU 

38.5 

2.305 

2.261 

1.551 


Table 22. GBN totals from the ME 

and RMSE values for the three ARW 

forecasts at the seven METAR and four buoy sites during 1-30 April 2012. 

1-30 April 2012 Totals 

Model 

Good 

Neutral 

Bad 

Ferrier-YSU 

27 

29 

23 

Lin-YSU 

45 

13 

22 

WDM6-YSU 

9 

35 

36 


3.4.2 Upper Air forecasts 

The AMU validated the three ARW configurations against the daily 0000 and 1200 UTC 
KWAL sounding. As with the fall forecasts, 12- and 24-hour upper air forecasts for the 0000 and 
1200 UTC initialized runs were validated with the corresponding KWAL sounding. The data 
were validated with the same vertical levels as in the fall forecasts. The average 30-day ME and 
RMSE were computed for wind direction, wind speed, T, and T d (Table 23). Based on the 
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results, Lin-YSU produced the best upper-air forecast followed by WDM6-YSU. Again, 
differences in the computed statistics were small with the exception of T d . 


Table 23. GBN chart of the average ME and RMSE for 1-30 April 2012 for the 
0000 and 1200 UTC KWAL RAOB for the three ARW configurations. 

30-day: Mean Error - KWAL RAOB 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

9.58 

0.052 

-0.490 

3.474 

Lin-YSU 

9.51 

0.030 

-0.479 

-0.026 

WDM6-YSU 

9.46 

0.046 

-0.483 

3.287 

30-day: RMSE - KWAL RAOB 

Model 

Wind Dir 

(deg) 

Wind Spd 

(m/s) 

Temp (F) 

Dewpt (F) 

Ferrier-YSU 

14.68 

3.358 

3.071 

10.026 

Lin-YSU 

14.45 

3.416 

3.103 

0.563 

WDM6-YSU 

14.25 

3.428 

3.144 

10.030 


3.4.3 Precipitation forecasts 

The AMU compared the 24-hour accumulation of observed rain using the NCEP Stage-ll 
analysis data to the 24-hr forecast accumulated rainfall from the 0000 and 1200 UTC model 
runs from 1-30 April 2012. The summary statistics of centroid distance (km), area ratio, and 
interest function from the MODE software are shown in Table 24. The results indicate that 
WDM6-YSU produced the best precipitation forecast, while Lin-YSU performed the worst. The 
Lin-YSU model did a better job of forecasting the fall season precipitation vs. the spring season 
precipitation. Since there were twice as many forecasts run in the spring season, more 
precipitation events were captured in the spring than in the fall. Therefore, it is more likely that 
the spring statistics are a more accurate representation of the skill of the model in predicting 
accumulated rainfall. In addition, once observational data is assimilated, individual performance 
of any model configuration may improve. 


Table 24. GBN chart for the average 30-day MODE statistics for 1-30 April 2012 

over the WFF WRF domain. 



30-day: MODE - Precipitation Statistics 

Model 

Centroid Distance (km) 

Area Ratio 

Interest 

Ferrier-YSU 

12.37 

0.75 

0.93 

Lin-YSU 

16.01 

0.69 

0.88 

WDM6-YSU 

10.85 

0.80 

0.92 


3.5 Recommendations and Future Work 

Fall and spring season results indicate that while all three ARW configurations performed 
similarly, both Ferrier-YSU and Lin-YSU performed slightly better than WDM6-YSU and better 
than the NMM configuration. Therefore, either Ferrier-YSU or Lin-YSU is the optimal model 
configuration for WFF. The AMU recommends running the WRF ARW over WFF using either 
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the Lin or Ferrier microphysical scheme and the YSU PBL scheme with a 4 km outer domain 
and a 1.33 km inner domain. 

The next step in this study will be to provide a recommended local DA and numerical 
forecast model design optimized for WFF to support space launch activities. The AMU will 
determine the best software and type of assimilation to use, as well as determine the best grid 
resolution for the initialization based on spatial and temporal availability of data and the wall 
clock run-time of the initialization. The AMU will transition from the WRF EMS to NU-WRF, a 
NASA-specific version of the WRF that takes advantage of unique NASA software and datasets. 
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4. Summary and Future Work 

This report summarizes the findings from an AMU task to determine the best model 
configuration for operational use at the ER and WFF to best predict winds, precipitation, and 
temperature. The AMU ran test cases in the warm and cool seasons at the ER and for the 
spring and fall seasons at WFF. For both the ER and WFF, the ARW core outperformed the 
NMM core. Results for the ER indicate that the Lin microphysical scheme and the YSU PBL 
scheme is the optimal model configuration for the ER. It consistently produced the best surface 
and upper air forecasts, while performing fairly well for the precipitation forecasts. Both the 
Ferrier and Lin microphysical schemes in combination with the YSU PBL scheme performed 
well for WFF in the spring and fall seasons. 

The AMU has been tasked with a follow-on modeling effort to recommended local DA and 
numerical forecast model design optimized for both the ER and WFF to support space launch 
activities. The AMU will determine the best software and type of assimilation to use, as well as 
determine the best grid resolution for the initialization based on spatial and temporal availability 
of data and the wall clock run-time of the initialization. The AMU will transition from the WRF 
EMS to NU-WRF, a NASA-specific version of the WRF that takes advantage of unique NASA 
software and datasets. 
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List of Acronyms 


AMU 

Applied Meteorology Unit 

ARW 

Advanced Research WRF 

CCAFS 

Cape Canaveral Air Force 
Station 

EMS 

Environmental Modeling 
System 

ER 

Eastern Range 

GFS 

Global Forecast System 

KSC 

Kennedy Space Center 

LIS 

Land Information System 

ME 

Mean Error 

MODE 

Method for Object-Based 
Diagnostic Evaluation 

MRF 

Medium-Range Forecast 

MYJ 

Mellor-Yamada-Janjic 

NAM 

North American Mesoscale 

NCAR 

National Center for 
Atmospheric Research 

NCEP 

National Centers for 
Environmental Prediction 

NMM 

Non-hydrostatic Mesoscale 
Model 


PBL 

Planetary Boundary Layer 

Ri 

Richardson number 

RMSE 

Root Mean Square Error 

RUC 

Rapid Update Cycle 

SOO 

Science and Operations Officer 

SPoRT 

Short-term Prediction 
Research and Transition 
Center 

SST 

Sea Surface Temperature 

STRC 

Science and Training 
Resource Center 

TKE 

Turbulent Kinetic Energy 

WDM6 

WRF Double-Moment 6-Class 

WFF 

Wallops Flight Facility 

WRF 

Weather Research and 
Forecasting 

WSM6 

WRF Single-Moment 6-Class 

WSR-88D 

Weather Surveillance Radar- 
1988 Doppler 

XMR 

CCAFS rawinsonde 3-letter 
identifier 

YSU 

Yonsei University 
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